Papers with multilayer perceptron (
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have shown remarkable results on various NLP tasks. |
| Approach: | They propose to improve the feed-forward network (FFN) in BERT with a higher computational cost than improving the multi-head attention (MHA). |
| Outcome: | The proposed model is 6.9 smaller and 4.4 faster than BERTBASE and has competitive performances on GLUE and SQuAD Benchmarks. |
LETS-C: Leveraging Text Embedding for Time Series Classification (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language modeling have shown promising results when applied to time series data. |
| Approach: | They propose a method to fine-tune large language models for time series classification tasks using text embedding models and a simple classification head. |
| Outcome: | The proposed model outperforms the current SOTA model on a time series classification benchmark and uses only 14.5% of the trainable parameters. |